<!DOCTYPE html>
<html >
<head>
    <meta charset="utf-8"/>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
    
    <title>拼接YUV420P图像 | xilixili.net</title>
    <meta name="renderer" content="webkit">
    <meta name="HandheldFriendly" content="True">
    <meta name="MobileOptimized" content="320">
    <meta name="apple-mobile-web-app-capable" content="yes">
    <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">

    <meta name="description" content="最近需要实现视频的左右拼接，从而可以实现2路摄像头的同时显示，调研使用了几种实现方式进行了对比。
ffmpeg方式谈到视频方面的处理，肯定首选 ffmpeg，想要实现上面的功能，其实 ffmpeg 就能实现，使用其 filter 就可以，大概思路是：

使用  pad 扩展出另外一个图像的空间
使用 overlay 复制另外一个图像到扩展出来的空间

整体实现不难，后续再讲一下这个，因为可以实现很">

    <meta name="twitter:card" content="summary">
    <meta name="twitter:title" content="拼接YUV420P图像 | xilixili.net">
    <meta name="twitter:description" content="最近需要实现视频的左右拼接，从而可以实现2路摄像头的同时显示，调研使用了几种实现方式进行了对比。
ffmpeg方式谈到视频方面的处理，肯定首选 ffmpeg，想要实现上面的功能，其实 ffmpeg 就能实现，使用其 filter 就可以，大概思路是：

使用  pad 扩展出另外一个图像的空间
使用 overlay 复制另外一个图像到扩展出来的空间

整体实现不难，后续再讲一下这个，因为可以实现很">

    <meta property="og:type" content="article">
    <meta property="og:title" content="拼接YUV420P图像 | xilixili.net">
    <meta property="og:description" content="最近需要实现视频的左右拼接，从而可以实现2路摄像头的同时显示，调研使用了几种实现方式进行了对比。
ffmpeg方式谈到视频方面的处理，肯定首选 ffmpeg，想要实现上面的功能，其实 ffmpeg 就能实现，使用其 filter 就可以，大概思路是：

使用  pad 扩展出另外一个图像的空间
使用 overlay 复制另外一个图像到扩展出来的空间

整体实现不难，后续再讲一下这个，因为可以实现很">

    
    <meta name="author" content="rangerlee">
    
    <link rel="stylesheet" href="/css/vno.css">
    <link rel="stylesheet" href="/css/font-awesome.min.css">

    
    <link rel="icon" href="/images/avatar-small.png">
    

    <meta name="generator" content="hexo"/>
    
    <link rel="alternate" type="application/rss+xml" title="xilixili.net" href="/atom.xml">
    

    <link rel="canonical" href="xilixili.net/2020/05/18/yuv420p-merge/"/>

                 
</head>

<body class="home-template no-js">
    <script src="//cdn.bootcss.com/jquery/3.2.1/jquery.min.js"></script>
    <script src="/js/main.js"></script>
    <span class="mobile btn-mobile-menu">
        <i class="fa fa-list btn-mobile-menu__icon"></i>
        <i class="fa fa-angle-up btn-mobile-close__icon hidden"></i>
    </span>

    
<header class="panel-cover panel-cover--collapsed">
  <div class="panel-main">
    <div class="panel-main__inner panel-inverted">
    <div class="panel-main__content">

        <h1 class="panel-cover__title panel-title"><a href="/" title="访问首页 xilixili.net">xilixili.net</a></h1>
        
        <span class="panel-cover__subtitle panel-subtitle"><!-- 生活と技术の杂记 --></span>
        
        <hr class="panel-cover__divider">
        
        <p class="panel-cover__description">生活と技术の杂记</p>
        
	<hr class="panel-cover__divider panel-cover__divider--secondary">

        <div class="navigation-wrapper">
          <div>
          <nav class="cover-navigation cover-navigation--primary">
            <ul class="navigation">
              <li class="navigation__item"><a href="/#blog" title="访问博客" class="blog-button">博客</a></li>
            
            </ul>
          </nav>
          </div>
          <div>
          <nav class="cover-navigation navigation--social">
  <ul class="navigation">

  <!-- Weibo-->
  

  <!-- Github -->
  
  <li class="navigation__item">
    <a href="https://github.com/rangerlee" title="查看我的GitHub主页" target="_blank">
      <i class="social fa fa-github"></i>
      <span class="label">Github</span>
    </a>
  </li>


<!-- Stack Overflow -->
        

  <!-- Google Plus -->
  

<!-- Facebook -->

  
<!-- Twitter -->

  

  <li class="navigation__item">
    <a href="/atom.xml" title="RSS" target="_blank">
      <i class="social fa fa-rss"></i>
      <span class="label">RSS</span>
    </a>
  </li>


  <li class="navigation__item">
    <a href="mailto:http://mail.qq.com/cgi-bin/qm_share?t=qm_mailme&amp;email=RzUmKSAiNSsiIgchKD8qJi4raSQoKg" title="邮件联系我" target="_blank">
      <i class="social fa fa-envelope"></i>
      <span class="label">Email</span>
    </a>
  </li>


  </ul>
</nav>

          </div>
        </div>

      </div>

    </div>

    <div class="panel-cover--overlay cover-alpha"></div>
  </div> 
</header>
<script type="text/javascript">
var side = document.getElementsByClassName("panel-cover")[0];
function checkWebp() {
	try{
		return(document.createElement('canvas').toDataURL('image/webp').indexOf('data:image/webp') == 0);
	}catch(err) {
		return false;
	}
};

if(checkWebp()) {
	side.style.backgroundImage = "url('/images/background-cover.jpg.webp')";
} else {
	side.style.backgroundImage = "url('/images/background-cover.jpg')";
}
</script>


    <div class="content-wrapper">
        <div class="content-wrapper__inner">
            <article class="post-container post-container--single">

  <header class="post-header">
    <div class="post-meta">
      <time datetime="2020-05-18T08:31:00.000Z" class="post-list__meta--date date">2020-05-18</time> &#8226; <span class="post-meta__tags tags">
  <a class="tag-link" href="/tags/技术/">技术</a>
 </span>
      <span class="page-pv">
       阅读 <span id="busuanzi_value_page_pv"><i class="fa fa-spinner fa-spin"></i></span>
      </span> 
   
    </div>
    <h1 class="post-title">拼接YUV420P图像</h1>
  </header>

  <section class="post">
    <p>最近需要实现视频的左右拼接，从而可以实现2路摄像头的同时显示，调研使用了几种实现方式进行了对比。</p>
<h3 id="ffmpeg方式"><a href="#ffmpeg方式" class="headerlink" title="ffmpeg方式"></a>ffmpeg方式</h3><p>谈到视频方面的处理，肯定首选 <code>ffmpeg</code>，想要实现上面的功能，其实 <code>ffmpeg</code> 就能实现，使用其 <code>filter</code> 就可以，大概思路是：</p>
<ul>
<li>使用  <code>pad</code> 扩展出另外一个图像的空间</li>
<li>使用 <code>overlay</code> 复制另外一个图像到扩展出来的空间</li>
</ul>
<p>整体实现不难，后续再讲一下这个，因为可以实现很多功能，但是对于我们目前的需求来说，这个功能致命之处在于 <code>ffmpeg</code> 需要两次处理才能实现，本机实测大概需要 110 毫秒，对于直播来说肯定是致命的。</p>
<p>因此就想到了另外一种方式，即手动进行内容拼接，毕竟两个图像大小相同且不需要其他处理。</p>
<h3 id="OpenCV方式"><a href="#OpenCV方式" class="headerlink" title="OpenCV方式"></a>OpenCV方式</h3><p><code>OpenCV</code> 也是处理图像的专业库，不过其使用的是自身的 <code>cv::Mat</code> 数据结构，以下为把 <code>ffmpeg</code> 解码出来的一帧进行复制拼接</p>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">SwsContext</span> *<span class="title">sws_ctx</span> = <span class="title">NULL</span>;</span></span><br><span class="line">sws_ctx = sws_getContext(avframe-&gt;width, avframe-&gt;height, </span><br><span class="line">                         (<span class="keyword">enum</span> AVPixelFormat)avframe-&gt;format,</span><br><span class="line">        		         w, h, AV_PIX_FMT_BGR24, SWS_BICUBIC,</span><br><span class="line">                         <span class="literal">NULL</span>, <span class="literal">NULL</span>, <span class="literal">NULL</span>);</span><br><span class="line"></span><br><span class="line">cv::Mat mat;</span><br><span class="line">mat.create(cv::Size(w, h), CV_8UC3);</span><br><span class="line">AVFrame *bgr24frame = av_frame_alloc();</span><br><span class="line">bgr24frame-&gt;data[<span class="number">0</span>] = (<span class="keyword">uint8_t</span> *)mat.data;</span><br><span class="line">avpicture_fill((AVPicture *)bgr24frame, bgr24frame-&gt;data[<span class="number">0</span>], AV_PIX_FMT_BGR24, w, h);</span><br><span class="line">sws_scale(sws_ctx,</span><br><span class="line">        (<span class="keyword">const</span> <span class="keyword">uint8_t</span>* <span class="keyword">const</span>*)avframe-&gt;data, avframe-&gt;linesize,</span><br><span class="line">        <span class="number">0</span>, avframe-&gt;height, <span class="comment">// from cols=0,all rows trans</span></span><br><span class="line">        bgr24frame-&gt;data, bgr24frame-&gt;linesize);</span><br><span class="line"></span><br><span class="line">av_free(bgr24frame);</span><br><span class="line">sws_freeContext(sws_ctx);</span><br></pre></td></tr></table></figure>
<p>这样就搞定了，不过还得将 <code>cv::Mat</code> 再转回 <code>AVFrame</code> 来回的数据交换也挺耗时的，放弃。</p>
<h3 id="手动方式"><a href="#手动方式" class="headerlink" title="手动方式"></a>手动方式</h3><p>手动方式就直接拼接，不过 <code>YUV</code> 格式和我们熟知的 <code>RGB</code> 格式并不相同，其使用 <code>Y</code> <code>U</code> <code>V</code> 三个分量，按其排列方式等可分为 <code>YUV420P</code> 等等，至于详细的介绍，请自行百度，将会明白下面的处理逻辑。</p>
<p>平时最常见的是 <code>YUV420P</code> 我们就以该格式为例说明</p>
<blockquote>
<p>网络上有不少相同例子，不过只有个别代码是对的，因为错误的例子没有考虑 AVFrame 的 linesize</p>
</blockquote>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i &lt; frame_height; i++) &#123; <span class="comment">//Y</span></span><br><span class="line">    <span class="built_in">memcpy</span>(frame-&gt;data[<span class="number">0</span>] + i * frame_width * <span class="number">2</span>, </span><br><span class="line">           frame1-&gt;data[<span class="number">0</span>] + i * frame1-&gt;linesize[<span class="number">0</span>], frame_width);</span><br><span class="line">    <span class="built_in">memcpy</span>(frame-&gt;data[<span class="number">0</span>] + frame_width + i * frame_width * <span class="number">2</span>, </span><br><span class="line">           frame2-&gt;data[<span class="number">0</span>] + i * frame2-&gt;linesize[<span class="number">0</span>], frame_width);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">uint8_t</span>* pU = frame-&gt;data[<span class="number">1</span>];</span><br><span class="line"><span class="keyword">for</span> (<span class="keyword">int</span> j = <span class="number">0</span>; j &lt; frame_height / <span class="number">2</span>; j++)&#123;</span><br><span class="line">    <span class="built_in">memcpy</span>(pU + (frame_width / <span class="number">2</span>) * j * <span class="number">2</span>, </span><br><span class="line">           frame1-&gt;data[<span class="number">1</span>] + frame1-&gt;linesize[<span class="number">1</span>] * j, frame_width / <span class="number">2</span>);</span><br><span class="line">    <span class="built_in">memcpy</span>(pU + (frame_width / <span class="number">2</span>) * (j * <span class="number">2</span> + <span class="number">1</span>), </span><br><span class="line">           frame2-&gt;data[<span class="number">1</span>] + frame2-&gt;linesize[<span class="number">1</span>] * j, frame_width / <span class="number">2</span>);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">uint8_t</span>* pV = frame-&gt;data[<span class="number">2</span>];</span><br><span class="line"><span class="keyword">for</span> (<span class="keyword">int</span> k = <span class="number">0</span>; k &lt; frame_height / <span class="number">2</span>; k++)&#123;</span><br><span class="line">    <span class="built_in">memcpy</span>(pV + frame_width / <span class="number">2</span> * k * <span class="number">2</span>, </span><br><span class="line">           frame1-&gt;data[<span class="number">2</span>] + frame1-&gt;linesize[<span class="number">2</span>] * k, frame_width / <span class="number">2</span>);</span><br><span class="line">    <span class="built_in">memcpy</span>(pV + frame_width / <span class="number">2</span> * (k * <span class="number">2</span> + <span class="number">1</span>), </span><br><span class="line">           frame2-&gt;data[<span class="number">2</span>] + frame2-&gt;linesize[<span class="number">2</span>] * k, frame_width / <span class="number">2</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>
<p>经测试使用和上面的 <code>ffmpeg</code> 采用相同的逻辑和数据，基本耗时在 10 毫秒左右，性能差距还是挺大的。</p>
<p>效果图：</p>
<p><picture class="picture"><source type="image/webp" srcset="/2020/05/18/yuv420p-merge/yuv.png.webp"><img class="image" src="/2020/05/18/yuv420p-merge/yuv.png" alt=""></picture> </p>
<h3 id="关于YUV保存"><a href="#关于YUV保存" class="headerlink" title="关于YUV保存"></a>关于YUV保存</h3><p>操作YUV之后，我们需要看结果，那么就需要将文件保存，保存方法如下</p>
<figure class="highlight cpp"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">SaveAvFrame</span><span class="params">(AVFrame *avFrame, <span class="keyword">const</span> <span class="keyword">char</span>* file)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">    FILE *fDump = fopen(file, <span class="string">"wb"</span>);</span><br><span class="line"></span><br><span class="line">    <span class="keyword">uint32_t</span> pitchY = avFrame-&gt;linesize[<span class="number">0</span>];</span><br><span class="line">    <span class="keyword">uint32_t</span> pitchU = avFrame-&gt;linesize[<span class="number">1</span>];</span><br><span class="line">    <span class="keyword">uint32_t</span> pitchV = avFrame-&gt;linesize[<span class="number">2</span>];</span><br><span class="line"></span><br><span class="line">    <span class="keyword">uint8_t</span> *avY = avFrame-&gt;data[<span class="number">0</span>];</span><br><span class="line">    <span class="keyword">uint8_t</span> *avU = avFrame-&gt;data[<span class="number">1</span>];</span><br><span class="line">    <span class="keyword">uint8_t</span> *avV = avFrame-&gt;data[<span class="number">2</span>];</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> (<span class="keyword">uint32_t</span> i = <span class="number">0</span>; i &lt; avFrame-&gt;height; i++) &#123;</span><br><span class="line">        fwrite(avY, avFrame-&gt;width, <span class="number">1</span>, fDump);</span><br><span class="line">        avY += pitchY;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> (<span class="keyword">uint32_t</span> i = <span class="number">0</span>; i &lt; avFrame-&gt;height / <span class="number">2</span>; i++) &#123;</span><br><span class="line">        fwrite(avU, avFrame-&gt;width / <span class="number">2</span>, <span class="number">1</span>, fDump);</span><br><span class="line">        avU += pitchU;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> (<span class="keyword">uint32_t</span> i = <span class="number">0</span>; i &lt; avFrame-&gt;height / <span class="number">2</span>; i++) &#123;</span><br><span class="line">        fwrite(avV, avFrame-&gt;width / <span class="number">2</span>, <span class="number">1</span>, fDump);</span><br><span class="line">        avV += pitchV;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    fclose(fDump);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>

  </section>

</article>

<section class="read-more">
           
    
               
            <div class="read-more-item">
                <span class="read-more-item-dim">最近的文章</span>
                <h2 class="post-list__post-title post-title"><a href="/2020/09/28/boost-property-ptree-json-write-chinese/" title="BOOST生成JSON中文乱码">BOOST生成JSON中文乱码</a></h2>
                <p class="excerpt">
                
                最近Linux生产环境在用boost库生成json的时候发现中文乱码，经调测发现是boost库的bug，在此记录。
boost生成Json一般用法boost有两个数据结构，其中一个是 property::ptree, 这个数据结构可以表示 xml json ini 等格式的文件内容，在不使用第三方比
                &hellip;
                </p>
                <div class="post-list__meta"><time datetime="2020-09-28T09:02:00.000Z" class="post-list__meta--date date">2020-09-28</time> &#8226; <span class="post-list__meta--tags tags">
  <a class="tag-link" href="/tags/技术/">技术</a>
</span><a class="btn-border-small" href="/2020/09/28/boost-property-ptree-json-write-chinese/">继续阅读</a></div>
                           
            </div>
        
        
               
            <div class="read-more-item">
                <span class="read-more-item-dim">更早的文章</span>
                <h2 class="post-list__post-title post-title"><a href="/2020/05/14/blockchain-fisco-bcos-dynamic-group/" title="FISCOBCOS~2.4动态群组试用">FISCOBCOS~2.4动态群组试用</a></h2>
                <p class="excerpt">
                
                续前文，近期 FISCO BCOS 2.4  版本发布了，最核心的特性是动态群组管理，这个功能免去了手动部署新群组的过程，极大的方便了群组管理。
新版本部署FISCO BCOS 2.4 版本部署前一版本基本没有区别，注意 2.3 版本部署的时候配置文件稍有变化（依然兼容以前配置）。按照以前的部署方法
                &hellip;
                </p>
                <div class="post-list__meta"><time datetime="2020-05-14T02:50:00.000Z" class="post-list__meta--date date">2020-05-14</time> &#8226; <span class="post-list__meta--tags tags">
  <a class="tag-link" href="/tags/技术/">技术</a>
</span><a class="btn-border-small" href="/2020/05/14/blockchain-fisco-bcos-dynamic-group/">继续阅读</a></div>
                       
            </div>
        
     
   
   
  
</section>



	<script>
	(function(){
		var bp = document.createElement('script');
		var curProtocol = window.location.protocol.split(':')[0];
		if (curProtocol === 'https') {
			bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';        
		}
		else {
			bp.src = 'http://push.zhanzhang.baidu.com/push.js';
		}
		var s = document.getElementsByTagName("script")[0];
		s.parentNode.insertBefore(bp, s);
	})();
	</script>



<script>
(function(){
var src = (document.location.protocol == "http:") ? "http://js.passport.qihucdn.com/11.0.1.js?8612a7fa26d7ba4562be733ae33649b1":"https://jspassport.ssl.qhimg.com/11.0.1.js?8612a7fa26d7ba4562be733ae33649b1";
document.write('<script src="' + src + '" id="sozz"><\/script>');
})();
</script>



            <footer class="footer">
    <span class="footer__copyright">
        &copy; 2023 rangerlee - 本站点采用 <a href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议</a>
       
    </span>
    <span class="footer__copyright">
             - 本站基于开源 <a href="http://hexo.io">Hexo</a> 搭建，采用 <a href="https://github.com/rangerlee/hexo-theme-new-vno">hexo-theme-new-vno</a> 主题，修改自 <a href="https://github.com/monniya/hexo-theme-new-vno ">new-vno</a> 主题  <a href="http://www.beian.miit.gov.cn">皖ICP备16009457号</a>
         </span>
       
    
    
</footer>


        </div>
    </div>

     
    
    <script>
        var _hmt = _hmt || [];
        (function() {
            var hm = document.createElement("script");
            hm.src = "//hm.baidu.com/hm.js?bab4f71524319c10819bcae280021624";
            var s = document.getElementsByTagName("script")[0]; 
            s.parentNode.insertBefore(hm, s);
        })();
    </script>



    <script async src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script>
    
    
</body>
</html>
